The number needed to treat: problems describing non-significant results.

The number needed to treat (NNT) and its counterpart, the number needed to harm (NNH), are commonly used measures of effect in studies of mental health treatments. The NNT is an absolute measure, which conveys a more direct and intuitive impression of the effect on an individual than the absolute risk reduction (ARR). To illustrate this, consider an imaginary trial comparing the effects of cognitive behavioural therapy versus interpersonal psychotherapy for people with bulimia. If 5% of people receiving cognitive behavioural therapy relapsed after treatment and 25% of people receiving psychotherapy relapsed, we could express this as an absolute risk reduction: ARR = absolute difference in event rates (25–5%) = 20% (interpretation: cognitive behavioural therapy reduces the absolute risk of relapse by 20% compared with psychotherapy) Alternatively, we could describe the result as a number needed to treat: NNT = 1/ARR (1/20%) = 5 (interpretation: treat five people with cognitive behavioural therapy rather than psychotherapy to prevent one relapse) The NNT allows us to dispense with percentage risks and helps describe effects in terms of real people. Confidence limits can also be provided for the NNT, which, loosely interpreted give us an idea of the range of effect we might expect most of the time. When the confidence interval includes a situation where intervention has no effect, the result is said to be non-significant. So far, so good. The NNT appears to be a simple measure that can intuitively convey both the magnitude and the certainty of a trial result. There is a complication when dealing with non-significant results, however. To understand how this complication comes about, let's look at how NNT behaves with respect to the ARR. The NNT is a reciprocal function; the inverse of absolute risk reduction. This relationship may be represented graphically (see webextra). The graph shows that the function is not continuous. As the ARR crosses the " line of no effect " from positive to negative , the NNT crosses from plus infinity to minus infinity. When the ARR is non-significant, by definition its confidence interval includes or touches zero. The confidence interval for the corresponding NNT will straddle plus and minus infinity. When the NNT describes a significant result, the interpretation is straightforward. Suppose our earlier result (ARR=20%) was conventionally significant at the 5% level. By definition, the 95% confidence interval for the ARR would not include zero. Suppose the confidence limits for …